{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1CmEDootEpNV"
      },
      "source": [
        "![Cohere-Logo-Color-RGB.png]()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_q-7fEZxmIHm"
      },
      "source": [
        "Large Language Models (LLMs) are becoming increasingly capable of comprehending text, among others excelling in document analysis. The new Cohere model, [Command-R](https://huggingface.co/CohereForAI/c4ai-command-r-v01), boasts a context length of 128k, which makes it particularly effective for such tasks. Nevertheless, even with the extended context window, some documents might be too lengthy to accommodate in full.\n",
        "\n",
        "In this cookbook, we'll explore techniques to address cases when relevant information doesn't fit in the model context window.\n",
        "\n",
        "We'll show you three potential mitigation strategies: truncating the document, query-based retrieval, and a \"text rank\" approach we use internally at Cohere.\n",
        "\n",
        "### Table of content:\n",
        "1. [Getting started](#getting-started)\n",
        "2. [Approach 1: Truncate](#truncate)\n",
        "3. [Approach 2: Query Based Retrieval](#query-based-retrieval)\n",
        "4. [Approach 3: Text Rank](#text-rank)\n",
        "\n",
        "### Summary\n",
        "\n",
        "| Approach     | Description               | Pros                                      | Cons                       | When to use?                       |\n",
        "|-----------------------|-------------------------------------------|-------------------------------------------|-------------------------------------------|-------------------------------------------|\n",
        "| Truncation            | Truncate the document to fit the context window.                    | - Simplicity of implementation<br>(does not rely on extrenal infrastructure)| - Loses information at the end of the document | Utilize when all relevant information is contained<br> at the beginning of the document. |\n",
        "| Query Based Retrieval| Utilize semantic similarity to retrieve text chunks<br> that are most relevant to the query. | - Focuses on sections directly relevant to<br> the query | - Relies on a semantic similarity algorithm.<br>- Might lose broader context | Employ when seeking specific<br> information within the text. |\n",
        "| Text Rank             | Apply graph theory to generate a cohesive set<br> of chunks that effectively represent the document. | - Preserves the broader picture.          | - Might lose detailed information. | Utilize in summaries and when the question<br> requires broader context. |\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Lwgm08jzah5c"
      },
      "source": [
        "<a id=\"getting-started\"></a>\n",
        "<a name=\"getting-started\"></a>\n",
        "# Getting Started"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "background_save": true,
          "base_uri": "https://localhost:8080/"
        },
        "id": "wuQ1PO8FadQf",
        "outputId": "c6cc36f7-6368-4cdc-ce57-4d95249e409f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Collecting cohere\n",
            "  Downloading cohere-4.54-py3-none-any.whl (52 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m52.8/52.8 kB\u001b[0m \u001b[31m1.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: aiohttp<4.0,>=3.0 in /usr/local/lib/python3.10/dist-packages (from cohere) (3.9.3)\n",
            "Collecting backoff<3.0,>=2.0 (from cohere)\n",
            "  Downloading backoff-2.2.1-py3-none-any.whl (15 kB)\n",
            "Collecting fastavro<2.0,>=1.8 (from cohere)\n",
            "  Downloading fastavro-1.9.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.1/3.1 MB\u001b[0m \u001b[31m33.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting importlib_metadata<7.0,>=6.0 (from cohere)\n",
            "  Downloading importlib_metadata-6.11.0-py3-none-any.whl (23 kB)\n",
            "Requirement already satisfied: requests<3.0.0,>=2.25.0 in /usr/local/lib/python3.10/dist-packages (from cohere) (2.31.0)\n",
            "Requirement already satisfied: urllib3<3,>=1.26 in /usr/local/lib/python3.10/dist-packages (from cohere) (2.0.7)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0,>=3.0->cohere) (1.3.1)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0,>=3.0->cohere) (23.2.0)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0,>=3.0->cohere) (1.4.1)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0,>=3.0->cohere) (6.0.5)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0,>=3.0->cohere) (1.9.4)\n",
            "Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0,>=3.0->cohere) (4.0.3)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.10/dist-packages (from importlib_metadata<7.0,>=6.0->cohere) (3.17.0)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.25.0->cohere) (3.3.2)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.25.0->cohere) (3.6)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.25.0->cohere) (2024.2.2)\n",
            "\u001b[31mERROR: Operation cancelled by user\u001b[0m\u001b[31m\n",
            "\u001b[0mTraceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/cli/base_command.py\", line 169, in exc_logging_wrapper\n",
            "    status = run_func(*args)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/cli/req_command.py\", line 242, in wrapper\n",
            "    return func(self, options, args)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/commands/install.py\", line 324, in run\n",
            "    session = self.get_default_session(options)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/cli/req_command.py\", line 98, in get_default_session\n",
            "    self._session = self.enter_context(self._build_session(options))\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/cli/req_command.py\", line 125, in _build_session\n",
            "    session = PipSession(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/network/session.py\", line 342, in __init__\n",
            "    self.headers[\"User-Agent\"] = user_agent()\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/network/session.py\", line 175, in user_agent\n",
            "    setuptools_dist = get_default_environment().get_distribution(\"setuptools\")\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/metadata/__init__.py\", line 75, in get_default_environment\n",
            "    return select_backend().Environment.default()\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/metadata/__init__.py\", line 63, in select_backend\n",
            "    from . import pkg_resources\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/metadata/pkg_resources.py\", line 8, in <module>\n",
            "    from pip._vendor import pkg_resources\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_vendor/pkg_resources/__init__.py\", line 3327, in <module>\n",
            "    def _initialize_master_working_set():\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_vendor/pkg_resources/__init__.py\", line 3301, in _call_aside\n",
            "    f(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_vendor/pkg_resources/__init__.py\", line 3339, in _initialize_master_working_set\n",
            "    working_set = WorkingSet._build_master()\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_vendor/pkg_resources/__init__.py\", line 620, in _build_master\n",
            "    ws = cls()\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_vendor/pkg_resources/__init__.py\", line 613, in __init__\n",
            "    self.add_entry(entry)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_vendor/pkg_resources/__init__.py\", line 669, in add_entry\n",
            "    for dist in find_distributions(entry, True):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_vendor/pkg_resources/__init__.py\", line 2130, in find_on_path\n",
            "    for entry in sorted(entries):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_vendor/pkg_resources/__init__.py\", line 2127, in <genexpr>\n",
            "    entries = (os.path.join(path_item, child) for child in safe_listdir(path_item))\n",
            "  File \"/usr/lib/python3.10/posixpath.py\", line 71, in join\n",
            "    def join(a, *p):\n",
            "KeyboardInterrupt\n",
            "\n",
            "During handling of the above exception, another exception occurred:\n",
            "\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/bin/pip3\", line 8, in <module>\n",
            "    sys.exit(main())\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/cli/main.py\", line 79, in main\n",
            "    return command.main(cmd_args)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/cli/base_command.py\", line 101, in main\n",
            "    return self._main(args)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/cli/base_command.py\", line 223, in _main\n",
            "    return run(options, args)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/cli/base_command.py\", line 206, in exc_logging_wrapper\n",
            "    logger.critical(\"Operation cancelled by user\")\n",
            "  File \"/usr/lib/python3.10/logging/__init__.py\", line 1524, in critical\n",
            "    self._log(CRITICAL, msg, args, **kwargs)\n",
            "  File \"/usr/lib/python3.10/logging/__init__.py\", line 1624, in _log\n",
            "    self.handle(record)\n",
            "  File \"/usr/lib/python3.10/logging/__init__.py\", line 1634, in handle\n",
            "    self.callHandlers(record)\n",
            "  File \"/usr/lib/python3.10/logging/__init__.py\", line 1696, in callHandlers\n",
            "    hdlr.handle(record)\n",
            "  File \"/usr/lib/python3.10/logging/__init__.py\", line 968, in handle\n",
            "    self.emit(record)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/utils/logging.py\", line 168, in emit\n",
            "    message = self.format(record)\n",
            "  File \"/usr/lib/python3.10/logging/__init__.py\", line 943, in format\n",
            "    return fmt.format(record)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/utils/logging.py\", line 119, in format\n",
            "    prefix += \" \" * get_indentation()\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pip/_internal/utils/logging.py\", line 69, in get_indentation\n",
            "    def get_indentation() -> int:\n",
            "KeyboardInterrupt\n",
            "^C\n",
            "Requirement already satisfied: tokenizers in /usr/local/lib/python3.10/dist-packages (0.15.2)\n",
            "Requirement already satisfied: huggingface_hub<1.0,>=0.16.4 in /usr/local/lib/python3.10/dist-packages (from tokenizers) (0.20.3)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface_hub<1.0,>=0.16.4->tokenizers) (3.13.1)\n",
            "Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from huggingface_hub<1.0,>=0.16.4->tokenizers) (2023.6.0)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from huggingface_hub<1.0,>=0.16.4->tokenizers) (2.31.0)\n",
            "Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface_hub<1.0,>=0.16.4->tokenizers) (4.66.2)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from huggingface_hub<1.0,>=0.16.4->tokenizers) (6.0.1)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface_hub<1.0,>=0.16.4->tokenizers) (4.10.0)\n",
            "Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.10/dist-packages (from huggingface_hub<1.0,>=0.16.4->tokenizers) (23.2)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface_hub<1.0,>=0.16.4->tokenizers) (3.3.2)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface_hub<1.0,>=0.16.4->tokenizers) (3.6)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface_hub<1.0,>=0.16.4->tokenizers) (2.0.7)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface_hub<1.0,>=0.16.4->tokenizers) (2024.2.2)\n",
            "\u001b[31mERROR: Operation cancelled by user\u001b[0m\u001b[31m\n",
            "\u001b[0mCollecting langchain\n",
            "  Downloading langchain-0.1.11-py3-none-any.whl (807 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m807.5/807.5 kB\u001b[0m \u001b[31m5.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: PyYAML>=5.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (6.0.1)\n",
            "Requirement already satisfied: SQLAlchemy<3,>=1.4 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.0.28)\n",
            "Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (3.9.3)\n",
            "Requirement already satisfied: async-timeout<5.0.0,>=4.0.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (4.0.3)\n",
            "Collecting dataclasses-json<0.7,>=0.5.7 (from langchain)\n",
            "  Downloading dataclasses_json-0.6.4-py3-none-any.whl (28 kB)\n",
            "Collecting jsonpatch<2.0,>=1.33 (from langchain)\n",
            "  Downloading jsonpatch-1.33-py2.py3-none-any.whl (12 kB)\n",
            "Collecting langchain-community<0.1,>=0.0.25 (from langchain)\n",
            "  Downloading langchain_community-0.0.27-py3-none-any.whl (1.8 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.8/1.8 MB\u001b[0m \u001b[31m7.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting langchain-core<0.2,>=0.1.29 (from langchain)\n",
            "  Downloading langchain_core-0.1.30-py3-none-any.whl (256 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m256.9/256.9 kB\u001b[0m \u001b[31m18.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting langchain-text-splitters<0.1,>=0.0.1 (from langchain)\n",
            "  Downloading langchain_text_splitters-0.0.1-py3-none-any.whl (21 kB)\n",
            "Collecting langsmith<0.2.0,>=0.1.17 (from langchain)\n",
            "  Downloading langsmith-0.1.23-py3-none-any.whl (66 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m66.6/66.6 kB\u001b[0m \u001b[31m7.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: numpy<2,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (1.25.2)\n",
            "Requirement already satisfied: pydantic<3,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.6.3)\n",
            "Requirement already satisfied: requests<3,>=2 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.31.0)\n",
            "Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (8.2.3)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.3.1)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (23.2.0)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.4.1)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (6.0.5)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.9.4)\n",
            "Collecting marshmallow<4.0.0,>=3.18.0 (from dataclasses-json<0.7,>=0.5.7->langchain)\n",
            "  Downloading marshmallow-3.21.1-py3-none-any.whl (49 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m49.4/49.4 kB\u001b[0m \u001b[31m5.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting typing-inspect<1,>=0.4.0 (from dataclasses-json<0.7,>=0.5.7->langchain)\n",
            "  Downloading typing_inspect-0.9.0-py3-none-any.whl (8.8 kB)\n",
            "Collecting jsonpointer>=1.9 (from jsonpatch<2.0,>=1.33->langchain)\n",
            "  Downloading jsonpointer-2.4-py2.py3-none-any.whl (7.8 kB)\n",
            "Requirement already satisfied: anyio<5,>=3 in /usr/local/lib/python3.10/dist-packages (from langchain-core<0.2,>=0.1.29->langchain) (3.7.1)\n",
            "Requirement already satisfied: packaging<24.0,>=23.2 in /usr/local/lib/python3.10/dist-packages (from langchain-core<0.2,>=0.1.29->langchain) (23.2)\n",
            "Collecting orjson<4.0.0,>=3.9.14 (from langsmith<0.2.0,>=0.1.17->langchain)\n",
            "  Downloading orjson-3.9.15-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m138.5/138.5 kB\u001b[0m \u001b[31m15.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1->langchain) (0.6.0)\n",
            "Requirement already satisfied: pydantic-core==2.16.3 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1->langchain) (2.16.3)\n",
            "Requirement already satisfied: typing-extensions>=4.6.1 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1->langchain) (4.10.0)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.3.2)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.6)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (2.0.7)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (2024.2.2)\n",
            "Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy<3,>=1.4->langchain) (3.0.3)\n",
            "Requirement already satisfied: sniffio>=1.1 in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3->langchain-core<0.2,>=0.1.29->langchain) (1.3.1)\n",
            "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3->langchain-core<0.2,>=0.1.29->langchain) (1.2.0)\n",
            "Collecting mypy-extensions>=0.3.0 (from typing-inspect<1,>=0.4.0->dataclasses-json<0.7,>=0.5.7->langchain)\n",
            "  Downloading mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)\n",
            "Installing collected packages: orjson, mypy-extensions, marshmallow, jsonpointer, typing-inspect, jsonpatch, langsmith, dataclasses-json, langchain-core, langchain-text-splitters, langchain-community, langchain\n",
            "Successfully installed dataclasses-json-0.6.4 jsonpatch-1.33 jsonpointer-2.4 langchain-0.1.11 langchain-community-0.0.27 langchain-core-0.1.30 langchain-text-splitters-0.0.1 langsmith-0.1.23 marshmallow-3.21.1 mypy-extensions-1.0.0 orjson-3.9.15 typing-inspect-0.9.0\n",
            "Requirement already satisfied: nltk in /usr/local/lib/python3.10/dist-packages (3.8.1)\n",
            "Requirement already satisfied: click in /usr/local/lib/python3.10/dist-packages (from nltk) (8.1.7)\n",
            "Requirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from nltk) (1.3.2)\n",
            "Requirement already satisfied: regex>=2021.8.3 in /usr/local/lib/python3.10/dist-packages (from nltk) (2023.12.25)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from nltk) (4.66.2)\n",
            "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (3.2.1)\n",
            "Collecting pypdf2\n",
            "  Downloading pypdf2-3.0.1-py3-none-any.whl (232 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m232.6/232.6 kB\u001b[0m \u001b[31m4.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hInstalling collected packages: pypdf2\n",
            "Successfully installed pypdf2-3.0.1\n"
          ]
        }
      ],
      "source": [
        "####################################################################################################\n",
        "#\n",
        "# Uncomment if you need to install the following packages\n",
        "#\n",
        "####################################################################################################\n",
        "# %%capture\n",
        "# !pip install cohere\n",
        "# !pip install python-dotenv\n",
        "# !pip install tokenizers\n",
        "# !pip install langchain\n",
        "# !pip install nltk\n",
        "# !pip install networkx\n",
        "# !pip install pypdf2"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UFi1QH3sZUEZ",
        "outputId": "3e72afe2-2c60-472e-b838-949676897e13"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Package punkt is already up-to-date!\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "False"
            ]
          },
          "execution_count": 51,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import os\n",
        "import requests\n",
        "from collections import deque\n",
        "from typing import List, Tuple\n",
        "\n",
        "import cohere\n",
        "\n",
        "import numpy as np\n",
        "\n",
        "import PyPDF2\n",
        "from dotenv import load_dotenv\n",
        "\n",
        "from tokenizers import Tokenizer\n",
        "\n",
        "import nltk\n",
        "nltk.download('punkt')  # Download the necessary data for sentence tokenization\n",
        "from nltk.tokenize import sent_tokenize\n",
        "\n",
        "import networkx as nx\n",
        "from getpass import getpass\n",
        "from IPython.display import HTML, display"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 17
        },
        "id": "6BtV3iJ_UQ2w",
        "outputId": "76303020-4c08-4f5e-aa5a-03762686455d"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "def set_css():\n",
        "  display(HTML('''\n",
        "  <style>\n",
        "    pre {\n",
        "        white-space: pre-wrap;\n",
        "    }\n",
        "  </style>\n",
        "  '''))\n",
        "get_ipython().events.register('pre_run_cell', set_css)\n",
        "\n",
        "set_css()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "K603hzyKda2f",
        "outputId": "611374e9-1e51-4012-c99c-9ecb17a17e19"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Enter your Cohere API key: ··········\n"
          ]
        }
      ],
      "source": [
        "# Set up Cohere client\n",
        "co_model = 'command-r'\n",
        "co_api_key = getpass(\"Enter your Cohere API key: \")\n",
        "co = cohere.Client(api_key=co_api_key)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "kI0inogLmIHp"
      },
      "outputs": [],
      "source": [
        "def load_long_pdf(file_path):\n",
        "    \"\"\"\n",
        "    Load a long PDF file and extract its text content.\n",
        "\n",
        "    Args:\n",
        "        file_path (str): The path to the PDF file.\n",
        "\n",
        "    Returns:\n",
        "        str: The extracted text content of the PDF file.\n",
        "    \"\"\"\n",
        "    with open(file_path, 'rb') as file:\n",
        "        pdf_reader = PyPDF2.PdfReader(file)\n",
        "        num_pages = len(pdf_reader.pages)\n",
        "        full_text = ''\n",
        "        for page_num in range(num_pages):\n",
        "            page = pdf_reader.pages[page_num]\n",
        "            full_text += page.extract_text()\n",
        "    return full_text\n",
        "\n",
        "def save_pdf_from_url(pdf_url, save_path):\n",
        "    try:\n",
        "        # Send a GET request to the PDF URL\n",
        "        response = requests.get(pdf_url, stream=True)\n",
        "        response.raise_for_status()  # Raise an exception for HTTP errors\n",
        "\n",
        "        # Open the local file for writing in binary mode\n",
        "        with open(save_path, 'wb') as file:\n",
        "            # Write the content of the response to the local file\n",
        "            for chunk in response.iter_content(chunk_size=8192):\n",
        "                file.write(chunk)\n",
        "\n",
        "        print(f\"PDF saved successfully to '{save_path}'\")\n",
        "    except requests.exceptions.RequestException as e:\n",
        "        print(f\"Error downloading PDF: {e}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Lw1ttnX_VM7N"
      },
      "source": [
        "In this example we use the Proposal for a Regulation of the European Parliament and of the Council defining rules on Artificial Intelligence from 26 January 2024, [link](https://data.consilium.europa.eu/doc/document/ST-5662-2024-INIT/en/pdf)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        },
        "id": "4JgYYTg7Shvq",
        "outputId": "bc24d5a7-8bde-4d2a-e569-8a6941612629"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "PDF saved successfully to 'example.pdf'\n",
            "Document length - #tokens: 128618\n"
          ]
        }
      ],
      "source": [
        "# Download the PDF file from the URL\n",
        "pdf_url = 'https://data.consilium.europa.eu/doc/document/ST-5662-2024-INIT/en/pdf'\n",
        "save_path = 'example.pdf'\n",
        "save_pdf_from_url(pdf_url, save_path)\n",
        "\n",
        "# Load the PDF file and extract its text content\n",
        "long_text = load_long_pdf(save_path)\n",
        "long_text = long_text.replace('\\n', ' ')\n",
        "\n",
        "# Print the length of the document\n",
        "print(\"Document length - #tokens:\", len(co.tokenize(long_text)))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wu9MjCfUmIHp"
      },
      "source": [
        "## Summarizing the text"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 17
        },
        "id": "PAa9xaU6-8ls",
        "outputId": "7e60436a-d8a2-42ec-df85-6ae3402bc106"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "def generate_response(message, max_tokens=300, temperature=0.2, k=0):\n",
        "  \"\"\"\n",
        "  A wrapper around the Cohere API to generate a response based on a given prompt.\n",
        "\n",
        "  Args:\n",
        "    messsage (str): The input message for generating the response.\n",
        "    max_tokens (int, optional): The maximum number of tokens in the generated response. Defaults to 300.\n",
        "    temperature (float, optional): Controls the randomness of the generated response. Higher values (e.g., 1.0) make the output more random, while lower values (e.g., 0.2) make it more deterministic. Defaults to 0.2.\n",
        "    k (int, optional): Controls the diversity of the generated response. Higher values (e.g., 5) make the output more diverse, while lower values (e.g., 0) make it more focused. Defaults to 0.\n",
        "\n",
        "  Returns:\n",
        "    str: The generated response.\n",
        "\n",
        "  \"\"\"\n",
        "  response = co.chat(\n",
        "    model = co_model,\n",
        "    message=message,\n",
        "    max_tokens=max_tokens,\n",
        "    temperature=temperature,\n",
        "    return_prompt=True\n",
        "    )\n",
        "  return response.text"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 17
        },
        "id": "80fgeyAhZDZD",
        "outputId": "3a756564-c6d9-44d6-ac97-ffb8c33d09f6"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "# Example summary prompt.\n",
        "prompt_template = \"\"\"\n",
        "## Instruction\n",
        "Summarize the following Document in 3-5 sentences. Only answer based on the information provided in the document.\n",
        "\n",
        "## Document\n",
        "{document}\n",
        "\n",
        "## Summary\n",
        "\"\"\".strip()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tq9Di1gImIHq"
      },
      "source": [
        "If you run the cell below, an error will occur. Therefore, in the following sections, we will explore some techniques to address this limitation.\n",
        "\n",
        "Error: :`CohereAPIError: too many tokens:`\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TECMEd6rdYN_"
      },
      "outputs": [],
      "source": [
        "prompt = prompt_template.format(document=long_text)\n",
        "# print(generate_response(message=prompt))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FLwHEBQojIXX"
      },
      "source": [
        "Therefore, in the following sections, we will explore some techniques to address this limitation."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AH-tKADHmIHq"
      },
      "source": [
        "<a name=\"truncate\"></a>\n",
        "# Approach 1 - Truncate"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OGxtSKUEmIHq"
      },
      "source": [
        "First we try to truncate the document so that it meets the length constraints. This approach is simple to implement and understand. However, it drops potentially important information contained towards the end of the document."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 17
        },
        "id": "JrvVls3smIHq",
        "outputId": "6ec69e1b-0d6d-4b96-e8ae-650ebc049a7a"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "# The new Cohere model has a context limit of 128k tokens. However, for the purpose of this exercise, we will assume a smaller context window.\n",
        "# Employing a smaller context window also has the additional benefit of reducing the cost per request, especially if billed by the number of tokens.\n",
        "\n",
        "MAX_TOKENS = 40000\n",
        "\n",
        "def truncate(long: str, max_tokens: int) -> str:\n",
        "    \"\"\"\n",
        "    Shortens `long` by brutally truncating it to the first `max_tokens` tokens.\n",
        "    This can break up sentences, passages, etc.\n",
        "    \"\"\"\n",
        "\n",
        "    tokenized = co.tokenize(long).token_strings\n",
        "    truncated = tokenized[:max_tokens]\n",
        "    short = \"\".join(truncated)\n",
        "    return short"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 124
        },
        "id": "bxBOLcMomIHq",
        "outputId": "0273fe8b-814b-4d5b-8da5-be85bfa9ea45"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "The document outlines the European Union's proposed Regulation on Artificial Intelligence, aiming to establish harmonised rules for AI development and use while ensuring fundamental rights protection. It defines the scope, purposes, and risks addressed, excluding national security and military purposes. Prohibited AI practices, such as real-time biometric identification in public spaces, are detailed. The regulation proposes a risk-based approach, classifying high-risk AI systems and setting requirements for providers and deployers. It establishes governance structures and obligations for general-purpose AI models, including transparency and copyright compliance. The document also covers issues like conformity assessment, responsibilities along the AI value chain, and standardisation. The regulation aims to foster trustworthy AI while protecting public interests and fundamental rights.\n"
          ]
        }
      ],
      "source": [
        "short_text = truncate(long_text, MAX_TOKENS)\n",
        "\n",
        "prompt = prompt_template.format(document=short_text)\n",
        "print(generate_response(message=prompt))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pC1l77odmIHr"
      },
      "source": [
        "<a name=\"query-based-retrieval\"></a>\n",
        "# Approach 2: Query Based Retrieval\n",
        "\n",
        "In this section we present how we can leverage a query retriereval based approach to generate an answer to the following question: `Based on the document, are there any risks related to Elon Musk?`.\n",
        "\n",
        "The solution is outlined below and can be broken down into four functional steps.\n",
        "\n",
        "1. Chunk the text into units\n",
        "    - Here we employ a simple chunking algorithm. More information about different chunking strategies can be found [here](TODO: link to chunking post).\n",
        "\n",
        "2. Use a ranking algorithm to rank chunks against the query\n",
        "    - We leverage another Cohere endpoint, `co.rerank` ([docs link](https://docs.cohere.com/reference/rerank-1)), to rank each chunk against the query.\n",
        "\n",
        "3. Keep the most-relevant chunks until context limit is reached\n",
        "    - `co.rerank` returns a relevance score, facilitating the selection of the most pertinent chunks. We can choose the most relevant chunks based on this score.\n",
        "\n",
        "4. Put condensed text back in original order\n",
        "    - Finally, we arrange the chosen chunks in their original sequence as they appear in the document.\n",
        "\n",
        "See `query_based_retrieval` function for the starting point.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rHwHtYSPjYKH"
      },
      "source": [
        "### Query based retrieval implementation"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 17
        },
        "id": "XIbGWw2GmIHr",
        "outputId": "9ca56d31-d7b4-4e48-e9d2-2279f9d2caaf"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "############################################################\n",
        "#\n",
        "# Utility functions for chunking\n",
        "#\n",
        "############################################################\n",
        "def split_text_into_sentences(text) -> List[str]:\n",
        "    \"\"\"\n",
        "    Split the input text into a list of sentences.\n",
        "    \"\"\"\n",
        "    sentences = sent_tokenize(text)\n",
        "\n",
        "    return sentences\n",
        "\n",
        "def group_sentences_into_passages(sentence_list, n_sentences_per_passage=5):\n",
        "    \"\"\"\n",
        "    Group sentences into passages of n_sentences sentences.\n",
        "    \"\"\"\n",
        "    passages = []\n",
        "    passage = \"\"\n",
        "    for i, sentence in enumerate(sentence_list):\n",
        "        passage += sentence + \" \"\n",
        "        if (i + 1) % n_sentences_per_passage == 0:\n",
        "            passages.append(passage)\n",
        "            passage = \"\"\n",
        "    return passages\n",
        "\n",
        "def build_simple_chunks(text, n_sentences=5):\n",
        "    \"\"\"\n",
        "    Build chunks of text from the input text.\n",
        "    \"\"\"\n",
        "    sentences = split_text_into_sentences(text)\n",
        "    chunks = group_sentences_into_passages(sentences, n_sentences_per_passage=n_sentences)\n",
        "    return chunks"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 160
        },
        "id": "gKS7Mo6pmIHr",
        "outputId": "ae67c468-b5d3-43a4-da36-83a2dc16956a"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Example sentence: ['This is to ensure that the deployer is aware and takes them into acc ount when using the  high - risk AI system.']\n",
            "\n",
            "Example passage: ['Notified bodies shall have procedures for the performance of activities which take due  account of the size of an undertaking, th e sector in which it operates, its structure, the  degree of complexity of the AI system in question. 8. Notified bodies shall take out appropriate liability insurance for their conformity  assessment activities, unless liability is assumed by the Member Sta te in which they are  established in accordance with national law or that Member State is itself directly  responsible for the conformity assessment. 9. Notified bodies shall be capable of carrying out all the tasks falling to them under this  Regulation with   the highest degree of professional integrity and the requisite competence  in the specific field, whether those tasks are carried out by notified bodies themselves or on  their behalf and under their responsibility. ']\n"
          ]
        }
      ],
      "source": [
        "sentences = split_text_into_sentences(long_text)\n",
        "passages = group_sentences_into_passages(sentences, n_sentences_per_passage=5)\n",
        "print('Example sentence:', np.random.choice(np.asarray(sentences), size=1, replace=False))\n",
        "print()\n",
        "print('Example passage:', np.random.choice(np.asarray(passages), size=1, replace=False))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 17
        },
        "id": "w6pjBAA9mIHr",
        "outputId": "ab3f3ae1-3b38-4999-f8f7-1259d070c88b"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "def _add_chunks_by_priority(\n",
        "    chunks: List[str],\n",
        "    idcs_sorted_by_priority: List[int],\n",
        "    max_tokens: int,\n",
        ") -> List[Tuple[int, str]]:\n",
        "    \"\"\"\n",
        "    Given chunks of text and their indices sorted by priority (highest priority first), this function\n",
        "    fills the model context window with as many highest-priority chunks as possible.\n",
        "\n",
        "    The output is a list of (index, chunk) pairs, ordered by priority. To stitch back the chunks into\n",
        "    a cohesive text that preserves chronological order, sort the output on its index.\n",
        "    \"\"\"\n",
        "\n",
        "    selected = []\n",
        "    num_tokens = 0\n",
        "    idcs_queue = deque(idcs_sorted_by_priority)\n",
        "\n",
        "    while num_tokens < max_tokens and len(idcs_queue) > 0:\n",
        "        next_idx = idcs_queue.popleft()\n",
        "        num_tokens += co.tokenize(chunks[next_idx]).length\n",
        "        # keep index and chunk, to reorder chronologically\n",
        "        selected.append((next_idx, chunks[next_idx]))\n",
        "    if num_tokens > max_tokens:\n",
        "        selected.pop()\n",
        "\n",
        "    return selected\n",
        "\n",
        "def query_based_retrieval(\n",
        "    long: str,\n",
        "    max_tokens: int,\n",
        "    query: str,\n",
        "    n_setences_per_passage: int = 5,\n",
        ") -> str:\n",
        "    \"\"\"\n",
        "    Performs query-based retrieval on a long text document.\n",
        "    \"\"\"\n",
        "    # 1. Chunk text into units\n",
        "    chunks = build_simple_chunks(long, n_setences_per_passage)\n",
        "\n",
        "    # 2. Use co.rerank to rank chunks vs. query\n",
        "    chunks_reranked = co.rerank(query=query, documents=chunks, model=\"rerank-english-v3.0\")\n",
        "    idcs_sorted_by_relevance = [\n",
        "        chunk.index for chunk in sorted(chunks_reranked, key=lambda c: c.relevance_score, reverse=True)\n",
        "    ]\n",
        "\n",
        "    # 3. Add chunks back in order of relevance\n",
        "    selected = _add_chunks_by_priority(chunks, idcs_sorted_by_relevance, max_tokens)\n",
        "\n",
        "    # 4. Put condensed text back in original order\n",
        "    separator = \" \"\n",
        "    short = separator.join([chunk for index, chunk in sorted(selected, key=lambda item: item[0], reverse=False)])\n",
        "    return short"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 17
        },
        "id": "SX04iD3emIHr",
        "outputId": "dbe55fc4-bbc4-482a-cace-f5e2fc9318ba"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "# Example prompt\n",
        "prompt_template = \"\"\"\n",
        "## Instruction\n",
        "{query}\n",
        "\n",
        "## Document\n",
        "{document}\n",
        "\n",
        "## Answer\n",
        "\"\"\".strip()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 321
        },
        "id": "53eSmzkJmIHr",
        "outputId": "093c1167-d66d-42f7-e65d-ffe02b2d71ff"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "The report discusses the regulation of biometric identification, specifically the use of real-time systems in publicly accessible spaces for law enforcement purposes. The use of these systems is generally prohibited except in certain circumstances, such as searching for missing people or identifying perpetrators of serious criminal offences.\n",
            "\n",
            "The report outlines a number of exceptions and additional prohibitions related to biometric identification. For instance, biometric categorisation based on specific beliefs or characteristics is generally prohibited, along with untargeted scraping of facial images for creating facial recognition databases.\n",
            "\n",
            "Additionally, the document mentions post-remote biometric identification, which is subject to additional safeguards. These include authorisation requirements and restrictions on their use for law enforcement purposes.\n",
            "\n",
            "The regulation also addresses the accuracy, robustness and cybersecurity of AI systems, including high-risk systems that involve biometric identification. Providers of such systems are responsible for ensuring compliance with these requirements.\n",
            "\n",
            "National competent authorities and market surveillance authorities are designated to oversee the implementation of these regulations. They have the power to enforce compliance, including the imposition of penalties for non-compliance.\n",
            "\n",
            "Overall, the report aims to establish a uniform legal framework for AI systems, especially those involving biometric identification, while ensuring the protection of fundamental rights and individual freedoms.\n"
          ]
        }
      ],
      "source": [
        "query = \"What does the report say about biometric identification? Answer only based on the document.\"\n",
        "short_text = query_based_retrieval(long_text, MAX_TOKENS, query)\n",
        "prompt = prompt_template.format(query=query, document=short_text)\n",
        "print(generate_response(message=prompt, max_tokens=300))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zfpvo9WwbB6e"
      },
      "source": [
        "<a name=\"text-rank\"></a>\n",
        "# Approach 3: Text rank"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YNbvzWqgmIHs"
      },
      "source": [
        "In the final section we will show how we leverage graph theory to select chunks based on their centrality. Centrality is a graph-theoretic measure of how connected a node is; the higher the centrality, the more connected the node is to surrounding nodes (with fewer connections among those neighbors).\n",
        "\n",
        "The solution presented in this document can be broken down into five functional steps:\n",
        "\n",
        "1. Break the document into chunks.\n",
        "    - This mirrors the first step in [Approach 2](#query-based-retrieval).\n",
        "\n",
        "2. Embed each chunk using an embedding model and construct a similarity matrix.\n",
        "    - We utilize `co.embed` [documentation link](https://docs.cohere.com/reference/embed).\n",
        "\n",
        "3. Compute the centrality of each chunk.\n",
        "    - We employ a package called [`NetworkX`](https://networkx.org/documentation/networkx-1.10/overview.html). It constructs a graph where the chunks are nodes, and the similarity score between them serves as the weight of the edges. Then, we calculate the centrality of each chunk as the sum of the edge weights adjacent to the node representing that chunk.\n",
        "\n",
        "4. Retain the highest-centrality chunks until the context limit is reached.\n",
        "    - This step follows a similar approach to [Approach 2](#query-based-retrieval).\n",
        "\n",
        "5. Reassemble the shortened text by reordering chunks in their original order.\n",
        "    - This step mirrors the last step in [Approach 2](#query-based-retrieval).\n",
        "\n",
        "See `text_rank` as the starting point.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zC_fkJgyj3Hv"
      },
      "source": [
        "### Text rank implementation"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 17
        },
        "id": "4fWR-HKpmIHs",
        "outputId": "1ddc2bf4-90d1-44df-d4cd-6908546db506"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "def text_rank(text: str, max_tokens: int, n_setences_per_passage: int) -> str:\n",
        "    \"\"\"\n",
        "    Shortens text by extracting key units of text from it based on their centrality.\n",
        "    The output is the concatenation of those key units, in their original order.\n",
        "    \"\"\"\n",
        "\n",
        "    # 1. Chunk text into units\n",
        "    chunks = build_simple_chunks(text, n_setences_per_passage)\n",
        "\n",
        "    # 2. Embed and construct similarity matrix\n",
        "    embeddings = np.array(\n",
        "        co.embed(\n",
        "            texts=chunks,\n",
        "            model=\"embed-english-v3.0\",\n",
        "            input_type=\"clustering\",\n",
        "        ).embeddings\n",
        "    )\n",
        "    similarities = np.dot(embeddings, embeddings.T)\n",
        "\n",
        "    # 3. Compute centrality and sort sentences by centrality\n",
        "    # Easiest to use networkx's `degree` function with similarity as weight\n",
        "    g = nx.from_numpy_array(similarities, edge_attr=\"weight\")\n",
        "    centralities = g.degree(weight=\"weight\")\n",
        "    idcs_sorted_by_centrality = [node for node, degree in sorted(centralities, key=lambda item: item[1], reverse=True)]\n",
        "\n",
        "    # 4. Add chunks back in order of centrality\n",
        "    selected = _add_chunks_by_priority(chunks, idcs_sorted_by_centrality, max_tokens)\n",
        "\n",
        "    # 5. Put condensed text back in original order\n",
        "    short = \" \".join([chunk for index, chunk in sorted(selected, key=lambda item: item[0], reverse=False)])\n",
        "\n",
        "    return short"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 17
        },
        "id": "YVVMr-S9mIHs",
        "outputId": "2fe871db-ac7a-4ac1-e7a1-e8c8292d95e4"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "# Example summary prompt.\n",
        "prompt_template = \"\"\"\n",
        "## Instruction\n",
        "Summarize the following Document in 3-5 sentences. Only answer based on the information provided in the document.\n",
        "\n",
        "## Document\n",
        "{document}\n",
        "\n",
        "## Summary\n",
        "\"\"\".strip()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 124
        },
        "id": "vyahJTGImIHs",
        "outputId": "0a12951a-b917-4e69-a5c4-951907e32a90"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <style>\n",
              "    pre {\n",
              "        white-space: pre-wrap;\n",
              "    }\n",
              "  </style>\n",
              "  "
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "The document outlines the European Union's Artificial Intelligence Act, which aims to regulate AI systems and models while promoting innovation. It establishes rules for developing, using, and monitoring AI, especially high-risk systems, to protect public interests and fundamental rights. The Act mandates risk management systems, data governance requirements, and transparency measures for high-risk AI. It also sets obligations for providers and deployers, including registration, and empowers competent authorities to enforce compliance. Additionally, it encourages codes of conduct and provides for AI regulatory sandboxes to support innovation. The EU will establish a database for high-risk AI systems, and the Commission will coordinate enforcement and promote best practices. Fines and penalties are proposed for non-compliance. The Act seeks to balance AI development and oversight, ensuring trustworthy and responsible use while fostering the EU's AI ecosystem.\n"
          ]
        }
      ],
      "source": [
        "short_text = text_rank(long_text, MAX_TOKENS, 5)\n",
        "prompt = prompt_template.format(document=short_text)\n",
        "print(generate_response(message=prompt, max_tokens=600))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "krJVT65vrRsX"
      },
      "source": [
        "## Summary\n",
        "\n",
        "In this notebook we present three useful methods to over come the limitations of context window size. In the following [blog post](TODO:add link), we talk more about how these methods can be evaluated."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "iUqiXtILroUY"
      },
      "outputs": [],
      "source": []
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.12.2"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}